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UNIFIED CAPTURE AND PROCESS INTERFACE 



NOTICE OF COPYRIGHT 

A portion of the disclosure of this patent document contains material that is 
subject to copyright protection. The copyright owner has no objection to the facsimile 
reproduction by anyone of the patent document or the patent disclosure, as it appears 
5 in the Patent and Trademark Office patent file or records, but otherwise reserves all 
copyright rights whatsoever. 

FIELD OF THE INVENTION 

The present invention relates generally to methods for manipulating a time 
10 based stream of information using a processing system, and more particularly to 
methods of capturing and editing a time based stream of information to create a 
presentation using a processing system. 

BACKGROUND 

15 There are many fields, such as entertainment, academics, business, health care, 

etc., that utilize media having time based streams of information. These time based 
streams of information may be in the form of full motion images, e.g. video and 
movies; audio, e.g. music, sound effects, and voice/narration; motion graphics, e.g. 
animation; still images, e.g. text, pictures; and combinations thereof. The information 

20 is usually comprised of a stream of individual units, e.g. frames or blocks. Frequently, 
this information is in a digital format and resides on any of a number of conventional 
electronic mediums. 

Oftentimes, the time based information is assembled into a presentation 
through the use of a processing system to edit the information. For example, a video 

25 may be edited and combined with audio, text, effects and/or graphics to create a 
presentation. A "presentation" is any changed version of an original time based 
stream of information or a modified copy of the original information. In addition, a 
"presentation" also may further be defined by at least one instruction for representing 
the original information in a changed form. In such an instruction-based presentation, 

30 the information is communicated in the manner according to the instruction. For 
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example, a software program may include directions on how a digital video should be 
played, for instance, where only certain frames of a video are chosen for display. 

Digital editing is typically performed in a processing system and involves 
random retrieval of information from storage, communication of the information, e.g. 
5 display on a monitor, and then manipulation of the information to produce the desired 
presentation. Various types of editing may be performed as new techniques for 
altering raw information are created. A portion or portions of sequential information, 
also referred to as a "clip," such as a continuum of segments, frame, block or footage, 
may be rearranged into a desired order. Typically, the presentation appears as a 

1 0 seamless flow of the edited information. Editing may further entail removing portions 
of clips. During editing, particular "edit features" may be added to the presentation, 
such as transitions between clips, special effects (e.g. black and white, brightness, 
contrast, texture, echoes, compositing, blurring, etc.), text, graphics, clack 
background, silent audio segments, combinations of different types of time based 

15 streams of information, such as audio and still images, and the like. Various types of 
editing may be performed as new techniques for altering raw information are created. 

Traditionally, editing of multimedia to produce presentations involved 
complex techniques performed by trained professionals. However, recently novice 
users have become interested in making presentations, as well. There has been 

20 increasing popularity in the processing of multimedia to produce sophisticated home 
movies, web sites, etc. 

Given the currently diverse level of skills among authors of presentations, it is 
desirable to provide processing a system that are easy and convenient to operate. Such 
system should require less hands-on activity of the user and have a simple user 

25 interface display. 

Although previous authoring systems provide for varying levels of ease for the 
novice user, there are still several drawbacks to these systems. Some prior systems 
include Studio DV from Pinnacle Systems Inc. of Santa Clara, California; Avid 
Cinema® from Avid Technology, Inc. of Tewksbury, Massachusetts; Adobe 

30 Premier® from Adobe of San Jose, California; Dazzle* Digital Video Creator™ from 
Dazzle Multimedia of Fremont, California; and DVgate™ from Sony Corporation of 
Tokyo, Japan. 
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In general, processing systems require that the time based stream of 
information be captured from a media source and entered into the system. The system 
must be able to receive information as quickly as it is being transferred from the 
source so as to not lose any portion of the information. The usual speed for loading 
5 video information into the system is about 3.5 Mbytes per second. The length of the 
video may vary, depending in part on the amount of storage space available in the 
system. 

In order to avoid missing the capture of any segments of information, earlier 
systems require that all other applications be suspended during the loading process. 

10 Furthermore, these prior systems usually disable all controls, such as keyboard 
commands, mouse commands and on-screen commands, while the system is in 
capture mode, and prepared for acquiring a time based stream of information. As a 
result, a user may not perform other tasks while information is being acquired. In fact, 
for some systems, the selection of a control by the user during this period disrupts the 

15 entire capture process. The user may stop capture in these prior systems by hitting any 
button on a keyboard, selecting an option on the display, clicking on a mouse, etc. 
Thus, the user must remain idle until after the information is obtained. During capture, 
the user cannot select from various controls or work with other applications on the 
system. As a result, much time is wasted during the capture process with these prior 

20 systems, and especially where a good deal of time is required to deposit large amounts 
of information. 

Some previous processing systems do permit display or output of the 
transferred information during the capture process for viewing or listening. However, 
the output by these systems is of low quality. Instead of devoting the system to 

25 generating high quality output, the system dedicates much of its resources to the 
reading and writing of information entering the system to avoid loss of information. 
For example, typical prior systems display at a rate of about 3 frames per second 
during video capture. However, the rate for video to transfer into a system is often 24 
to 30 frames per second. As a result, the display during the capture process is a poor 

30 representation of the quality of the information that is actually being captured. The 
user is not able to tell the true quality of information entering the system until after the 
information is stored. 

Moreover, prior systems generally have complicated user interfaces for 
capture and edit of the time based stream of information. The systems require separate 
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display screens for the various modes of operation, such as acquiring of information 
and editing of the information. In order to read and write the information into the 
system, the user must select a capture mode, whereupon the system displays a capture 
screen and the tools necessary for acquiring information. Thereafter, the user switches 
5 to an edit mode for display of an edit screen having controls for manipulating the 
information. This toggling between multiple screens for edit and capture complicates 
the process and wastes screen real estate. 

In light of the shortcomings of the various currently available systems, there is 
still a need for creating a presentation of a time based stream of information in a 

10 manner that is quick and easy to perform. In particular, there is a desire for a 
processing system that has a simple user interface that provides efficient use of screen 
real estate by providing a single screen that is functional for both capture and edit. 
The output of information being captured within the system should also substantially 
reflect the quality of the incoming information. Furthermore, there is a need for a 

15 system that permits a user to perform multiple tasks at once, such as run capture 
routines simultaneous with other applications. The system should reduce the need for 
hands-on by the user, such as the necessity for manually directing various operations. 
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SUMMARY OF THE INVENTION 

In one embodiment of the present invention, a method for collecting a time 
based stream of information in a processing system for generating a presentation is 
provided. The method includes communicating with a time based stream of 
information source, presenting capture information from the time based stream of 
information on a first portion of a display, and process information for constructing 
the presentation on a second portion of the display. At least one enabled control 
element is also presented for editing the information, performing side operations, and 
directing other tasks of the processing system. An output control may also be 
presented for controlling the display of the time based stream of information. 

The method may further include capturing the time based stream of 
information from the information source. The system may present capture information 
automatically in response to communicating with the information source. The capture 
information may include a capture output that is provided on the display at the same 
rate or substantially the same rate as the transfer rate for the time based stream of 
information. In some cases, the capture is by the execution of an interrupt procedure. 
Often, such an interrupt procedure iterates at the same or substantially the same rate 
as the transfer rate of the time based stream of information. Furthermore, an edit 
output may be provided on the same portion of the display that the capture output is 
presented. 

In another embodiment of a method for collecting the time based stream of 
information, the system detects an information source that in communication with the 
processing system. Automatically, in response to detecting the source, capture 
information from the time based stream of information is provided on a display. At 
times, the processing system may perform automatic checks for the time based stream 
of information source in communication with the processing system. The detection is 
by receiving a signal from the information source having a time based stream of 
information, through a capture port on the processing system. The system may capture 
the time based stream of information and further display capture output at the same 
rate or substantially the same rate as the transfer rate for the time based stream of 
information. 

Another embodiment of a method for generating a presentation of a time based 
stream of information includes capture of the time based stream of information during a 
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capture mode and presenting a capture output on a viewing portion of a display. Furthermore, 
during an edit mode, an edit output is presented on the viewing portion of the display. A 
capture output may be provided at the same rate or substantially the same rate as the transfer 
rate for the time based stream of information. In some cases, at least one enabled control 
5 element is also provided on the display during the capture mode and edit mode. Such enabled 
control element maybe for performing side operations. 

In order to generate the presentation of a time based stream of information, a 
processing system may be employed which has a capture port for acquiring the time based 
stream of information, a storage medium, a display device and a processor. The processor is 
10 for communicating with an information source through the capture port. The processor is 
further configured to present capture information on a first portion of the display device, 
process information on a second portion of the display device and at least one enabled control 
element, e.g. for editing or performing side operations. Usually, the processor is also used for 
capturing the time based stream of information from the information source, such as through 
15 execution of an interrupt procedure that may iterate at the same or substantially the same rate 
as the transfer rate of the time based stream of information. The processor may further assist 
in presenting a capture output at the same or substantially the same rate as the transfer rate for 
the time based stream of information. In some embodiments, the processor may be for 
automatically presenting capture information in response to communication with the 
20 information source. 

According to still further embodiments, a computer readable medium may be 
provided that has stored therein a plurality of sequences of instructions, which, when 
executed by a processing system, cause the processor to perform a variety of processes to 
related to collection the time based stream of information and generation of a presentation. 
25 The instructions command the processor to communicate with a time based stream of 
information source, provide capture information on a first portion of a display, process 
information for constructing the presentation on a second portion of the display, and at least 
one enabled control element. Additional sequences of instructions may cause the processor to 
capture the time based stream of information or provide an edit output on the first portion of 
30 the display after the presenting of capture information. Of course, other embodiments may 
provide only the instructions themselves or methods of performing these steps. 

The processing system may also have a means for communicating with an 
information source having a time based stream of information, means for presenting capture 
information from the time based stream of information on a first portion of the display 
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device, means for presenting process information for constructing the presentation on a 
second portion of the display device, and means for presenting at least one enabled control 
element. 

The benefits of collecting a time based stream of information by the processing 
5 system are direct in that efficient use of screen real estate are presenting by a single screen 
that is functional for both capture and edit, as well as other side operations, i.e. multitasking, 
during the capture process. By using the present system, especially through the interrupt 
procedure, multiple tasks may be performed virtually simultaneous with the capture process. 
Moreover, the system has the ability to present a close to accurate representation of the 
1 0 information being acquired without compromising the capture process. 

Other features and advantages of these and other embodiments are discussed 
in detail below. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



The present invention is illustrated by way of example, and not limitation, in 
the figures of the accompanying drawings in which: 

Figure 1 illustrates one embodiment of an authoring environment in which a 
5 time based stream of information may be processed, in accordance with the teachings 
presented herein. 

Figure 2 is a block diagram of one embodiment of a processing system, 
configured in accordance with the teachings presented herein. 

Figure 3 illustrates a flow chart, which shows a method for communicating 
10 with an information source, in accordance with the present processes. 

Figures 4A-4B are flow diagrams of methods of reading and writing a time 
based stream of information into a processing system, wherein Figure 4A shows a 
prior art process to capture the information and Figure 4B shows a process to capture 
the information and perform other routines in accordance with the present methods. 
15 Figures 5A-5C are examples of user interfaces for displaying both capture 

information and process information, according to one embodiment of the present 
invention, wherein Figure 5A is a GUI showing a representation of an acquired time 
based stream of information, Figure 5B is a GUI showing controls for editing text 
into a presentation and Figure 5C is a GUI showing controls for editing transitions 
20 into a presentation. 

Figure 6 is a block diagram of a machine-readable medium storing executable 
code and/or other data to provide one or a combination of mechanisms for processing 
a time based stream of information, configured in accordance with one embodiment 
of the present invention. 

25 
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DETAILED DESCRIPTION 



The processing system described below is an authoring tool that is configured 
to provide simple and convenient acquisition and editing of a time based stream of 
information in the system creating a presentation. One embodiment of the system 
incorporates a feature for automatically sensing an information source having a time 
based stream of information and for activating a capture mode. Another configuration 
of the system includes a single user interface that is functional for both a capture 
mode of operation and an edit mode of operation. The system may provide for high 
quality output of information being acquired. 

In still another embodiment, the user interface includes enabled controls so 
that various tasks may be performed while the information is being imported into the 
system. Although the system performs only one computer instruction at a time, it may 
multitask by alternating the programs or sets of instructions it performs. During 
capture of the time based stream of information, the system simply takes turns 
managing the programs that the user effectively starts. The system operates at speeds 
that make it seem as though the user's tasks are being performed simultaneous with 
information capture and without the system losing any of the received information. 
The ability of the system to permit a user to perform various procedures while 
capturing can save the user time. 

The time based stream of information and related presentation created by use 
of the processing system may be of any form, such as full motion images with audio, 
full images without audio, audio only, still images, motion graphics, e.g. a succession 
of photos or graphics, or any combination thereof. 

Hardware Overview 
Figure 1 illustrates an exemplary authoring environment 10 in which a time 
based stream of information may be processed to produce a presentation. A 
processing system 12, according to one embodiment of the present invention, is in 
communication with an information source 80 that has a time based stream of 
information at a capture port 14 (i.e. information collection interface) and optionally 
through an intermediate digital converter 82. Processing system 12 may read, copy, 
manipulate, store, delete, output, print and/or transfer information provided by source 
80. An optional external player 84 receives communication from processing system 
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12, either indirectly from a transfer port 16 and through pathway 86, and/or directly 
through a removable storage medium 18. Although Figure 1 demonstrates one layout 
of authoring environment 10, the scope of the present invention anticipates any 
number of information sources and players, coupled and arranged in various fashions 
and configured in accordance herewith. 

The information source 80 is any type of device that generates and transfers 
data or data signals related to the time based stream of information. The information 
source 80 may be a photographic or full motion image capture interface such as a 
camera, camcorder, digital camera, etc.; a sound capture interface, such as a laser disc 
player (e.g. CD player), tape player/recorder, MP 3 player, microphone, etc.; a 
keyboard; a scanner; a computer; a server; a computer program; the Internet, a sensor; 
any one of numerous available data collection devices; etc. 

The information source may generate digital data signals and be 
communicatively coupled directly to the processing system 12. Alternatively, the 
information source may generate signals in a format other than digital, e.g. analog 
signals. As shown in Figure 1, such non-digital data source may send signals to a 
digital converter 82 for changing the signals into a digital format prior to transfer of 
the data to the processing system 12. In another configuration, the information source 
80 resides in the processing system, such as a software file having data that is 
imported to an editing application program employed by the present invention. For 
example, a graphic image may be designed from scratch using a graphics program and 
may be imported to the editing application for use in the presentation. 

Communication between the information source 80 and the processing system 
12 is often through means of a FireWire® (from Apple Computer, Inc. of Cupertino, 
California), iLink or IEEE 1394 communication protocol. Where a FireWire brand 
protocol is employed, capture port 14 in the processing system may be a six-pin 
FireWire brand port. Also, a corresponding four-pin FireWire brand port may be 
present on the digital converter 82, where a digital converter is included, and/or on the 
information source 80. In this case, a six-pin-to-four-pin cable may be used to connect 
the ports of the digital converter 82 or information source 80 and processing system 
12. 

Other communication schemes are also suitable for transfer of data from 
information source 80 and/or digital converter 82 to the processing system 12, such as 
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an Ethernet connection (i.e., capture port 14 may be an Ethernet port), serial 
interfaces, parallel interfaces, RS422 and/or RS432 interfaces, Livewire interfaces, 
Appletalk busses, small computer system interfaces (SCSI), ATM busses and/or 
networks, token ring and/or other local area networks, universal serial buses (USB), 
PCI buses and wireless (e.g., infrared) connections, Internet connections, and other 
communication links for conveying the time based stream of information from an 
information source 80 to the processing system 12. In addition, source 80 may store 
the information on a removable storage source, which is coupled to, e.g. inserted into, 
the processing system 12 and in communication with the capture port 14. For 
example, the source 80 may be a tape, CD, hard drive, disc or other removable storage 
medium. 

Processing system 12 is any device configured to receive the time based 
information and manipulate the information to generate a presentation, e.g. a 
computer system or workstation. In one embodiment, the processing system 12 
includes a platform 20, e.g. a personal computer (PC), such as a Macintosh® (from 
Apple Computer, Inc. of Cupertino, California), Windows®-based PC (from 
Microsoft Corporation of Redmond, Washington), or one of a wide variety of 
hardware platforms that runs the UNIX operating system or other operating systems. 
Often, the processing system 12 is configured to send the resulting presentation to an 
external player 84. 

The external player 84 may be an IP host, a computer, a personal digital 
assistance (PDA), a server, a tape/CD/MP3 player, or other device that is configured 
to receive presentation information from processing system 12 and to display the 
presentation. In one instance of an authoring environment, the external player 84 is 
the same device as the information source 80, such as a camcorder. In still another 
case, no external player is provided and the presentation is played on the processing 
system and/or projected from the processing system onto a medium, e.g. a screen. 

The external player 84 may receive information directly through removable 
storage 18 from the processing system 12. Removable storage 18 represents any of a 
number of detachable storage mediums, e.g. magnetic, optical, magneto-optical, tape, 
and/or other type of machine-readable medium or device for storing information. For 
example, the removable storage 18 may be a hard disk, a read-only or writeable 
optical CD (e.g. CD ROM, DVD), a disc, tape, etc. 
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External player 84 may also receive presentation information indirectly from 
the processing system 12, such as through a pathway 86. The communication pathway 
86 may be through various networking mechanisms, such as a FireWire brand 
connection (or iLink or IEEE 1394 type of connection), LAN, WAN, telephone line, 
5 serial line Internet protocol (SLIP), point-to-point protocol (PPP), an XDSL link, a 
satellite or other wireless link, a cable modem, ATM network connection, an ISDN 
line, a DSL line, Ethernet, or other communication link between the processing 
system and external player. 

The processing system has components for handling time based information in 

10 various ways. As shown in Figure 2, these components include an optional detector 
24 for sensing communication with the information source 80 through capture port 
14, a display 70 for showing a user interface and the time based stream of 
information, storage 30 for holding the information, processor 26 for manipulating the 
information and possibly a transfer port 16 or removable storage 18 for sending the 

15 information so processed. Upon review of this specification, it will be appreciated by 
those skilled in the art that the components of processing system 12 may be connected 
in various ways in addition to those described herein. 

Now referring in more detail to the components shown in Figure 2, processing 
system 12 includes processor 26, which may represent one or more processors. Some 

20 exemplary processors are a Motorola Power PC processor, an Intel Pentium (or x86) 
processor, etc. The processor 26 may also be a microprocessor. The processor 26 runs 
an operating system and applications software that controls the operation of other 
system components. The processor may be configured to perform multitasking of 
several processes at the same time. 

25 Storage 30 is coupled to processor 26 by a bus 22. The storage 30 may be a 

dynamic random access memory (DRAM) and/or may include static RAM (SRAM). 
Typically the storage is a hard disk drive, but may alternatively be another magnetic 
disk, a magneto-optical disk or other read/write device. The processor may also be 
coupled to other types of multiple storage areas that may be considered as part of the 

30 storage 30 or separate from the storage 30. Some alternative storage areas may 
include cache, flash memory, etc., and which may also be considered the removable 
storage 18. Usually storage 30 has a capacity in the megabyte to gigabyte range, and 
more typically, at least 2 gigabytes capacity and can sustain throughput of at least 3.5 
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megabytes per second of information. However, as technology advances, the capacity 
of the storage may increase. 

The storage 30 contains the acquired time based stream of information within 
files as well as executable code that provides functionality for processing the time 
5 based stream of information in accordance with the present invention. The time based 
stream of information is stored within files in an information storage area 50. The 
storage may also include separate other storage areas for storing various presentation 
related information, such as references to clips and the modified presentation. 

Bus 22 further couples the processor 24 to an input/output (I/O) control 90, a 

10 transfer port 16 and/or removable storage 18. The I/O control 90 commands one or 
multiple I/O device(s) 92. Some conventional I/O devices are key boards, 
mouses/trackballs or other pointing devices, microphones, speakers, magnetic disk 
drives, optical disk drives, printers, scanners, digital cameras, etc. 

In one configuration of processing system, a means for communicating with 

15 an information source that has a time based stream of information is provided. For 
example, bus 22 additionally couples detector 24 to the processor 26. The processing 
system may detect a connection between the processing system and the information 
source through detector 24. 

The detector 24 may perform an inquiry as to whether an information source is 

20 in communication with the processing system. Typically, the detector determines a 
connection with the information source by sensing particular signals, such as 
electrical current output, that are released from the information source, and usually 
when the source is energized, i.e. "turned-on." Such signals may be in various forms 
such as digital or analog. The detector informs the operating system of the presence 

25 of a source. The detector system may be further configured to conduct periodic checks 
for communication with the information source, where an inquiry is sent to the 
operating system as to whether communication with a source has been found. This 
polling for a source may take place at any predetermined intervals, e.g. every one-half 
second. 

30 Figure 3 is a flowchart that illustrates one method of the processing system 

detecting an information source. The system initially provides a display with enabled 
controls 100. Prior to communication with the information source, the system is 
usually configured for the edit mode. The detector checks if there is a signal from the 
information source 102. If such signal is not sensed, the detector informs the 
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operating system. The processing system then queries the operating system if it has 
noticed an information source. Where a source had not been detected, the processing 
system pauses for a predetermined time 106 before checking for the information 
source again. However, if at any time, the detector processing system receives an 
input check command 104, such as from the user, the detector processing system will 
check for the information source without waiting through the pause period. Once a 
node, i.e. the information source, is added to capture port 14 and the node is detected, 
the system ceases to perform the routine inspections for the information source and 
connection with the information source for receiving information is opened. 

The detector 24 may provide for automatic detection where the detector senses 
the coupling of the system with the information source and automatically engages a 
capture mode. This automatic detection may be activated from an editing window. 
This editing window has controls for manipulating the time based stream of 
information. While still showing the editing window, the detector instructs the 
processor 26 to automatically switch to a capture mode of operation and execute a 
software routine for capturing the information. During the capture mode the 
processing system is prepared to read, store and display the time based information. 
The system begins capturing the stream of information 108 which may be monitored 
in the editing window. Various capture information, process information and enabled 
controls are provided on the system display 70. Furthermore, toggle controls may be 
presented to permit a user to switch between edit mode and capture mode, usually 
within the same window, e.g. editing window. 

In one example, the communication between the system and with the 
information source or digital converter is through means of a Fire Wire brand protocol 
and the detector 24 receives the FireWire type of signals that have been transmitted 
from the information source through the FireWire brand cables. The FireWire brand 
protocol is capable of providing sequential frames of the time based stream of 
information. The detector indicates to the processor that communication with an 
information source exists and the processor prepares the system for reading and 
writing of the time based information. The detector further senses when the 
information source is no longer in communication with the processing system by the 
detector failing to receive the source signal. 
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In one embodiment, the processing system remains in an edit mode during 
loading of the time based stream of information. Thus, the edit and capture modes 
simultaneously run. 

However, the system may also be manually switched between capture and edit 
modes. The user may toggle between modes of operation by activating either a 
capture mode control and edit mode control on the display. Where a user attempts to 
capture information and no information source is in communication with the system 
as detected by the detector, the user may be informed that a source is not present. 

Further to the details of platform 20 in Figure 2, a display control 74 is 
coupled to the processor 24 also through bus 22. The display control 74 
communicates with the display 70 in the processing system 12 to couple the display 
70 to the platform 20. Display 70 may be any one of a number of conventional display 
devices such as a liquid crystal display, cathode ray tube, plasma display, video 
display, or other type of display device. For sharp colors and grayscale, display 70 
may be an SVGA monitor, with a resolution of 800 x 600 (26 dpi) or better, with an 
active area of at least 1 5 inches, and may be controlled using a true color video card. 

The present invention additionally involves at least one and usually a plurality 
of control elements 72, e.g. objects, that may be visible on the display or invisible. An 
enabled control element may be used to control the capture process and/or edit 
process, and optionally to perform side operations that are related or unrelated to the 
generation of a presentation. Control elements that are associated with the capture or 
edit processes allow a user to manipulate and design a presentation or otherwise 
control it. 

The processing system may contain any number of different types of control 
elements. A control element may be activated by the user or by other means to create 
control commands, where the control element responds by sending instructions to the 
processor. For example, a processing control element conveys directions to the 
processor regarding how a clip is to be treated or displayed. Each control element 
includes properties that describe how it will react when selected by the user or other 
means. Control elements that are used for side operations instruct the processor to 
execute applications or tasks other than capture and edit functions, e.g. during 
multitasking. For example, a control element may send instructions to the processor to 
execute network services, such as e-mail applications and, web browsers. Other side 
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operations may be word-processing programs, spreadsheet programs, detection or 
communication with external devices/media (such as recognition of a CD ROM), etc 

In the present invention, the control elements may be either enabled or 
disabled during capture. However, in prior processing systems, control elements are 
disabled during the capture process. These previous systems employ sequential 
routines, commonly used by those skilled in the art, for capturing, processing of 
control commands and operating other applications. 

Means are provided for capturing the time based stream of information from 
the information source. Figure 4A shows a prior sequential-type method 150 for 
capturing time based information in which an application procedure 152 is executed 
when the processing system is in capture mode. In one stage of the application 
procedure 152, signals from an information source is captured 154. The system reads 
the information, shows the information on the display and writes the information to a 
storage medium. However, when control elements are activated, the system moves 
away from the capture step and switches to another stage that includes a process 
control commands loop 156 for handling the control instructions. The CPU of a 
system employing such a sequential-type method is dedicated to processing the 
control commands. While application procedure 102 is in the process control 
command loop 156, the capturing operation ceases and the processing system stops 
communicating with the information source. During this period, the information that 
is released from the information source does not get loaded into the system. Rather, 
the processing of control commands continues through to completion, at which time, 
the capturing step 154 may continue. Thus, capture of information and other 
applications cannot be simultaneously executed with these sequential-based systems, 
hi order to acquire the information and avoid processing of control commands, the 
system must disable the control elements. 

By comparison to prior capture techniques, a low-level interrupt-type method, 
according to the present invention, effectively permits intermittent processing of 
control element commands while, at the same time, loading the time based 
information. With the interrupt-type method, in addition to the capture application, a 
user may run any software applications, e.g. network communication protocols, such 
as Appletalk, Internet or e-mail applications, word processing applications, etc. 

According to one embodiment of the present invention, as shown in Figure 
4B, interrupt-type method 160 includes an application procedure 162 that may be 
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executed when the processing system is in capture mode 164 and when it is not in 
capture mode, e.g. when the system is in edit mode. Where the processing system is 
operating in a non-capture mode, the application procedure conducts normal 
operations, such as processing of control commands 180 from control elements. 
However, when the capture mode is indicated and the system is capturing the 
stream(s) of information, the interrupt procedure 166 fires the appropriate capture 
tasks and causes a temporary suspension of the application procedure 162 m an 
asynchronous manner. Thus, the processing of control commands proceeds until the 
capture process interrupts the other application tasks. 

This interrupt procedure 166 is performed at periodic intervals as a separate 
procedure from the application procedure 162. The capture steps of the interrupt 
procedure are certain to iterate at regular times, despite the activities, e.g. processing 
of control commands, of the application procedure. 

The interrupt procedure 166 includes a quick reading of incoming information 
168, e.g. the current frame of the time based stream of information that is being 
received from the information source. This current frame may be copied into a proxy 
170. The current frame from the proxy may be shown 172 on the display in a step 
that may or may not be outside of the interrupt procedure 166. In a slower step of the 
interrupt procedure, the information is written to storage 174. The information written 
into storage maybe viewed and further edited by use of the control elements. 

The essential capture steps are included in the interrupt procedure because 
these instructions are ensured to be carried out. The interrupt procedure is performed 
at its regularly scheduled intervals despite any other activity of the application 
procedure, e.g. showing of information. For example, writing to storage is vital to the 
capture process and, therefore, are included in interrupt procedure. Optional steps, on 
the other hand, may or may not be included in the interrupt procedure, but may be 
performed by the applications procedure instead. For example, the showing of 
capture information to the user may be excluded from the interrupt procedure. The 
operations of the application procedure are not certain to be performed. For example, 
where the user selects a control element, the application procedure may skip from 
showing the captured information to the processing control command loop. However, 
when the next iteration of interrupt procedure is to occur, the steps in the application 
procedure are over-ridden by the interrupt procedure. In-between iterations, the 
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application procedure continues with its tasks, e.g. showing of capture information or 
processing of control commands. 

Usually, the processing system according to the present invention, outputs the 
information, e.g. depicts the information on the display, at a rate (the "output rate") 
5 that is substantially similar to the rate in which each unit of the information, e.g. 
frame of video, is arriving from the information source (the "transfer rate"). In 
contrast with prior systems, where low quality displaying (i.e. considerably lower than 
the transfer rate, such as 30 to 70% of the transfer rate) is used to devote more of the 
system to capture, the present invention may use high quality (i.e. the same or 

10 substantially similar to the transfer rate, such as greater than 70% and more usually 93 
to 100% of the transfer rate) because all of the information is certain to be stored by 
the interrupt procedure. The speed at which the information is displayed depends on 
numerous factors, such as side operations being simultaneously performed by the 
system, e.g. network access and CD ROM detection; image quality, e.g. based on 

15 decompression schemes employed; and processor speed or graphic acceleration. For 
example, where the transfer rate is about 30 frames per second (fps), the 
corresponding output rate of the present invention is usually 29 to 30 fps. The 
transfer rate depends on the particular transfer rate standard employed, as described in 
detail below. The resulting capture output is of high quality that is similar to the 

20 quality of the information residing in the information source. 

The interrupt procedure is advantageous in that the process is very quickly 
performed and rapidly repeats. The time for performance is less than the transfer rate 
of the information entering the system. 

The repetitions of the interrupt procedure occur at a rate that is substantially 

25 similar to or the same as the transfer rate. Thus, all of the information may be read 
and stored while still processing control commands in between each of the iterations 
of the interrupt procedure. As a result, the user does not recognize the interruptions 
and the capture process as well as other instructions appear to be simultaneously 
executed by the system. Moreover, the interrupt procedure guarantees that 

30 information is not omitted from the reading step because the iterations of the interrupt 
procedure and length of time to perform are timed to correspond with the rate of 
information transfer from the information source. 

The transfer rate of the information entering the system is usually the same or 
substantially the same speed that the information runs prior to entering the system, 
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such as the rates established by the video standards of various regions. For example, 
movie film usually runs with a speed of 24 fps. Some video follows the NTSC 
(standard used in North America and Japan), where the video speed is 29.97 to 30 fps. 
Hence for transfer of NTSC video, the interrupt procedure iterations are typically at 
5 least every 1/29 to 1/30 second and more usually 1/29.97 second to 1/30 second. For 
PAL systems (standard used in parts of Europe, China and Hong Kong), the frame 
rate is 25 fps and thus interrupt procedure is substantially at least every 1/25 second. 
Other standards may be employed, such as SECAM. 

In any case, the interrupt procedure usually should iterates at the transfer rate, 

1 0 close to the transfer rate or slightly faster than the transfer rate, e.g. within at least 
l/10tfi second of the transfer rate and more typically within l/100th second. The 
iterations should not be slower than the transfer rate and the iterations are usually at 
the rate that corresponds to the frame rate of video standard. 

During the capture process, the time based stream of information is written 

15 into storage to be referenced while creating the presentation. For convenience, the 
stored information may be compressed to conserve storage space. Generally, 
compression formats are either high efficiency or low efficiency and either loss-less 
or lossy. Lossy compression schemes are characterized by components of an original 
image being absent from a reconstructed image after a compression-decompression 

20 cycle, whereas loss-less schemes do not drop any information. 

Low compression schemes (i.e., those that do not provide significant 
compression ratios) that may be used include joint photographic experts group (JPEG) 
compression schemes that use Fourier analysis-based methods, such as the discrete 
cosine transform, to compress data; graphics interchange format (GIF) compression 

25 schemes, which use LZW algorithms; bitmapped image compression schemes and 
tagged image file format (TEFF) compression schemes, etc. Alternatively, high 
efficiency compression schemes, such as wavelet, motion wavelet, Motion Picture 
Experts Group (MPEG) motion JPEG, Lempel Ziv and/or fractal compression 
schemes, etc., may be employed. Use of high efficiency schemes may be preferred 

30 where storage space is limited. For example, wavelet compression schemes may be 
20 times more efficient than JPEG compression schemes, providing for a more 
detailed representation of the data at the same bit budget. Once the time based 
information is captured and stored, it may be referenced for editing and generating a 
presentation. 
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The stored information is often outputted as , process information. Output may 
be through various mechanisms, such as depiction on the display or play through 
speakers Means for presenting the capture information, process information and 
control elements are provided. 

Figure 5 A, depicts an exemplary graphical user interface (GUI) 230 for 
displaying both capture information and process information on the display 50. 
Capture information is related to the time based information being acquired by the 
system while in capture mode. The capture information may include a depiction or 
replay of the time based stream of information (referred to as a "capture output") 232, 
as well as other characteristics of the information that is being acquired. The capture 
output sets forth the current frames that are being received and read, such as newly 
acquired information that is present in a proxy, i.e. buffer, prior to being written into 
storage. A time code 237 may be provided as capture information that indicates the 
length of a clip. In addition, the capture information may include an audio synch time 
line 242 (depicted under video time line 140) on which references to clips 236 (as 
described below) are arranged and depicted in size according to the length of the clip 
in storage. A user may opt for the audio synch time line to automatically display the 
references for each incoming stream of information. For example, the user may 
choose a preferences control from a menu and select to have the references appear in 
the audio synch time line rather than the shelf portion of the user interface. The 
capture information may further include a free space monitor 234 to show the amount 
of storage space remaining or amount of space consumed by stored information. 
Capture information may also include tools, e.g. control elements, for controlling the 
capture process. 

The process information that is displayed on the GUI 230 may include an edit 
output to reveal stored clips of the time based information for processing/editing 
during an edit mode. The edit output depicts how the clips could appear in 
presentation after editing is complete. For example, the user may desire to modify a 
clip, such as add a title or transition to a clip. The user may observe the edit output of 
the clip with the selected title or transition and the user may choose to accept or reject 
the modification as part of the presentation. 

The process information may also include at least one object reference 236, 
e.g. icon, with each reference 236 being associated with a clip of the time based 
information contained in storage 30. Typically, the reference 236 is displayed as a 
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"thumbnail" in the form of a still image of the first frame of the associated clip. The 
reference may be provided on a shelf 238 portion of the GUI 230. The number of 
references that may be displayed on the shelf depends, inter alia, on the resolution of 
the display, e.g. 9 to 12 references. By selecting the associated reference and 
5 employing other process information, individual clips may be edited. 

Other process information displayed on the GUI 230 may also include at least 
one control element for the user to manipulate the time based information and 
construct a presentation, at least one time line that represents the edited presentation, 
such as video time line 240 and audio time line 242, and other elements that add 

1 0 convenience to the edit process. 

The present invention frequently has a single GUI to displayboth capture 
information and process information. Such a GUI is functional for both capture and 
edit procedures, thus avoiding the necessity of switching to multiple GUI's to view 
and work with information relating to capture and information relating to the 

15 processing of the presentation. Any convenient portions of the screen may be used to 
show the capture and process information without the user needing to toggle between 
screens. The GUT may include toggle controls 340 to switch between modes of 
operation, usually within the same window. When capture made control 342 is 
activated, the system is prepared to capture information and when edit mode control 

20 342 is triggered, the system is ready for editing of the information. 

In one embodiment, the same portion of the display may be used to show 
capture information and process information at different times. In one such 
embodiment, a capture output may be presented on a viewing portion of the display 
during capture of the time based information and an edit output may be presented on 

25 the same viewing portion during editing of the presentation. In the alternative, a 
portion of the display provides capture information, such as the capture output, and 
another portion of the display contains process information. An example of this 
embodiment is where capture information is shown in a window on the display, and 
process information in at least one other window on the display. 

30 Typically, one or more windows are present in the GUI, which may be opened 

by processor 24 or by the user. One such user interface illustrated in Figure 5A has a 
monitor window 246. During the capture mode, the monitor window 246 may exhibit 
the capture output of the time based information. This monitor window may be the 
viewing portion of the display that shows the capture output and edit output. 
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In addition to showing the capture output, the monitor window 246 may depict 
the edit output. Usually within the same window, the capture output is shown while 
the information is acquired and the edit output is shown while editing the 
presentation. By this dual use of the monitor window, the screen area is conserved and 
5 the user interface is simplified. Various other windows may be present on the display 
of the processing system to provide at least one control element. 

The use of a single GUI to display functional, i.e. enabled, control elements 
for capture, processing, and side operations, is especially useful while the system is 
capturing the time based information from the information source of information. 

10 Such enabled control elements permit a user to perform various operations while the 
processing system is capturing the time based stream of information. 

One set of control elements are output controls 248 that implement how the 
user chooses to view or listen to the information from the system. Output control 
elements may direct the showing of the information on the monitor window 246, the 

15 playing of information through audio speakers, or output through other convenient 
devices in communication with the system. There are several types of output controls 
248 that may be provided by the processing system. 

The output controls 248 may be used to move the play-head (i.e. pointer to 
indicate the location in the information available for use) to particular sections of the 

20 information stored in the storage. One such output control is a play control 250 that 
directs the system to play or show the information. Other kinds of output controls 
include a rewind control 252 and a forward control 254. The play- head may be jogged 
backward by the user depressing the rewind control 252 to shuttle the play-head in 
reverse direction. The output of information is advanced by the user selecting the 

25 forward control 254 during playback to shift forward at faster than the normal play 
rate, e.g. double the speed. Also provided by the processing system may be a stop 
control 256 for ceasing the output action and a pause control 258 for temporarily 
suspending the movement of the information from the system. In practice, the stop 
control 256 may be selected and then the rewind control 252 or forward control 254 

30 selected to quickly skip forward or backwards through the footage. 

In one method of outputting the time based information through the use of 
output controls, the play control 250 is activated and then the rewind control 252 or 
forward control 254 chosen to move to a position that is a immediately prior to the 
appropriate position for capture. The stop control 256 is selected and the play control 
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250 is reactivated. When the scene of interest is displayed or a particular portion of 
audio is played, the clip may be edited. 

Several additional output controls may also be employed for alternative ways 
of communicating the information. For example, a full screen mode control 252 may 
be provided to permit the presentation to be shown in a window that spans the entire 
area of the display. Also, some control elements may adjust the quality of playback. 
A preferences option with a playback tab (not shown), and an edit pull down menu 
254 are some optional quality controls. In addition, a sound control knob 256 
functions to adjust the intensity of audio. 

A smooth motion control may be present to allow the user to create a smoother 
playback than would be typically expected. Usually, the bandwidth necessary to 
display digital video requires that visual quality be compromised in order to deliver a 
high number of frames per second (fps) during playback. Also, a better image control 
may be included to provide a crisper playback and more detail from each frame 
displayed. Such option for a better image is often at the expense of motion quality. 
The number of frames per second that may be displayed is dependent on the 
processing power of the system, where higher power results in higher frame rate and 
playback that is less choppy. 

Some forms of control elements that are visible on the screen are buttons, 
"pop-up" or pull-down menus, scroll bars and iconic images. Other control elements 
may be invisible to the screen and thus are not included on the GUI. There are a 
variety of interactive mechanisms to activate the control elements, such as keyboard 
input, voice command, mouse manipulation, text entry, pen-to-text data entry device, 
touch screen, network signals, satellite transmissions, preprogrammed triggers within 
the system, instructional input from other applications, etc. All of the control elements 
described herein are by way of example and are not intended to limit the choices that 
are or may become available in the art. 

Other control elements are not visible on the screen but may be present on 
input/output devices, such as a keyboard that is in communication with the I/O 
control. For example, a space bar on a keyboard may also be depressed to start 
capture and depressed again to stop capture of the information on the fly while 
playing the time based information during capture mode. 

The table shown below describes other control elements that may be present 
on the keyboard, where the items in the "action" column describe the command 



Application 



23 



Ref. #004860.P2471 



activated by depressing the keyboard button and the items in the "shortcut" column 
are the corresponding keyboard buttons. 



Keyboard shortcuts Navigation 
Action Shortcut 



Play/Stop and Start/Stop Capture Space Bar 



Play-head to Beginning of Home 
Movie 



Play-head to End of Movie End 



Forward One Frame 



Right Arrow 



Forward Ten Frames 



Shift + Right Arrow 



Roll Play-head Forward Hold Down Right Arrow 



Search Forward (fforward) Command + ] 



Back One Frame 



Left Arrow 



Back Ten Frames 



Shift + Left Arrow 



Roll Play-head Backward Hold Down Left Arrow 



Selection 
Action 



Search Backward (rewind) Command + [ 



Shortcut 
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Multiple Selection Shift + Click Item 

Multiple Selection (range) Click 1 st item then Shift + Click last item 

Moving/Cropping 
Action Shortcut 

Move Audio Clip Click Clip + Left or Right Arrow 

Move Audio Clip Ten Click Clip, then Shift +Left or Right Arrow 
Frames 

Move Crop Marker Click Marker + Left or Right Arrow 

Move Crop Marker Ten Click Marker, 
Frames Shift +Left or Right Arrow 

Accept/Cancel 
Action Shortcut 

Accept Dialogs (OK) Return 

Cancel Dialogs Esc or Command + C 

Cancel Rendering 

Volume 
Action 

Increase System Volume 
Decrease System Volume 
Titling 

Application 



then 



Command + . (period) 



Shortcut 

Up Arrow 

Down Arrow 
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Action 



Shortcut 



Move Between Text Fields Tab 



File Menu Functions 
Action 

New Project 
Open Project 



Save Project 



Export Movie 



Save Frame As 
Import File 
Quit 



Edit Menu Functions 
Action 

Undo 
Redo 



Cut 



Copy 



Paste 



Shortcut 



Command + N 



Command + O 



Command + S 



Command + E 



Command + F 



Command + 1 



Command + Q 



Shortcut 

Command + Z 

Shift + Command + Z 
Command + X 



Command + C 



Command + V 
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Crop Command + K 

Split Clip at Play-head Command + T 



Select All Command + A 



S elect None Command + D 



Help Menu Functions 
Action Shortcut 

iMovie Help Command + ? or Help key 

Processing control elements are useful for editing and generating the 
presentation shown variously in the GUI 230 in Figures 5A, 5B and 5C. The user 
may manipulate the clip representations 236 in forming segments of a presentation. In 
one embodiment, a representation is moved to various portions of the display by 
moving a cursor over the representation that may be on a shelf 238 (as shown in 
Figure 5A), a video time-line 240, an audio time-line 242 or other portion of the 
display and dragging the representation to another portion of the display. Other 
functionalities may be present to relocate the representations such as mouse clicks, 
keyboard commands, control elements, etc. 

The representation may be moved by the user to the monitor window 246 so 
that the clip may be viewed on the monitor window. A monitor window time line 260 
depicts where the play-head 262 as it travels along the course of the clip. A variety of 
output controls 248 may be used to move the play-head to various positions on the 
clip. When the play-head is at the frame of interest, the pause control 258 may be 
activated and the frame edited. 

Particular segments of information are aligned in the order that they will 
appear in the presentation. Each clip that is to be included in the presentation is 
positioned in the presentation by arranging the clips representation along a video time 
line 240 or audio time line 242. 
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The chosen clip may be edited by use of the edit box 268 (as shown in Figure 
5A) that includes various editing options, such as titles control 270, transitions control 
300, music control 330 and sound control 350. 

Title control 270 includes instructions for adding text, such as headings, labels 
5 or captions, to a clip. Selection of the title control 270 opens a title window 274 as 
shown in Figure 5B. A title time line 276 shows the length of the selected clip and the 
position of the play-head 278. The user may navigate the play-head 278. The user 
may navigate the play-head to various frames in the clip by positioning a cursor over 
the play-head through use of a mouse, or using other cursor positioning means. The 
10 mouse may be used by clicking and holding down on it and sliding the mouse to the 
left or right along the title time-line relative to the desired location of the play-head. 
The title window 274 frequently also includes one or more text field(s) 280 for 
describing the desired text. A position menu 282 permits the user to choose where and 
how the text is to appear on the frame. For example, the text may be a centered title, 
15 appear to bounce through the length of the frame, centered with multiple text lines, 
centered as a title, letters flying across the frame, words flying, music video, rolling 
center credits, rolling credits, scroll with a pause, scrolling as a block, typewriter, and 
the like. A font menu 288 may be used to dictate the character type. An "over black" 
control 284 may be used to insert an additional black frame at the play-head position 
20 and the title over the black frame. An example of a reference element with text over 
black 285, having the title, "THE END" is shown on time line 240. On the other hand, 
a "color" control 286 may be used to overlay the text over the current frame. The text 
may be shown in a preview window 290 by selecting a preview control 292. Preview 
simulates the edited reference without the edited version being actually saved in 
25 storage. An add control 294 may be provided for storing the reference with the title. 
The reference with edit feature 296 may then be shown on the monitor window 246. 
The initial clip 01 maybe split into the reference with edit feature 296 and remaining 
portion reference 298, as shown on video time line 240. 

A transition control 300 has instructions for inserting an effect between clips, 
30 where selection of the transition control 300 opens a transition window 302, as 
depicted in Figure 5C. A transition effects menu 304 permits the user to choose from 
any one of numerous transition options. Examples of transitions are cross dissolve, 
cross dissolve slow, fade in, fade out, overlap, push right, slowly push right, scale 
down, slowly scale down, and the like. The frame location of the play-head 314 along 
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transition time ruler 316 may be positioned by the same or similar manner as 
described above for introducing titles in a clip. The transition is typically inserted as a 
frame immediately after the play-head position by reference element with transition 
352 and then appears on visual time line 240. At the point of the inserted transition 
5 frame 352, the clip appears to split with clip 01 reference 320 representing the portion 
of the clip preceding the transition and clip 02 reference 322 indicating the clip 
immediately after the transition frame 352, as shown on video-time line 240. 

Furthermore, the exemplary edit box 268 shown in Figure 5 A includes music 
control 330 for incorporating music tracks to the presentation where the music may be 

10 recorded by controls in the music control window. Sound control 350 in the edit box 
268 is for including sound effect to portions of the presentation. Usually, the sound 
effects are prerecorded and stored within the processing system. Some examples of 
sounds are cat meow, dog bark, crickets, crowd applause, trumpets fanfare, wild 
laugh, water lapping, whistle, thunder, horn honk and footsteps. 

15 The presentation, as edited, may be saved in a variety of formats, such as a 

QuickTime file or other formats suitable for transfer via e-mail, web pages, CD- 
ROM, etc. The presentation may be accessed at a later time for additional editing or 
may be transferred from the processing system, such as conveyed to an external 
player. 

20 In addition to the features described above, the processing system may further 

include alternative aspects that enhance ease and convenience of use. For example, 
referring again to Figure 1, the information source 80 may be controlled through the 
processing system 12. Source controls operate the information source 80, usually 
where no digital converter 82 is employed as an intermediate between the information 

25 source and processing system. The source controls may be presented on the GUI. In 
one embodiment, the output controls 248, as shown in Figure 2 also serve as the 
source controls to direct the monitor window views at one time and send action 
commands to the information source 80 at another time. Thus, the source controls 
may include rewind control 252, forward control 254, play control 250, pause control 

30 258 and stop control 256. 



Software Overview 
Various software components may be provided within or in communication 
with the system that cause the processor to execute the numerous methods employed 
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in creating the presentation. Figure 6 is a block diagram of a machine-readable 
medium storing executable code and/or other data to provide one or a combination of 
mechanisms for collecting and manipulating the time based stream of information, 
according to one embodiment of the invention. The machine-readable storage medium 
400 represents one or a combination of various types of media/devices for storing 
machine-readable data, which may include machine-executable code or routines. As 
such, the machine-readable storage medium 400 could include, but is not limited to 
one or a combination of a magnetic storage space, magneto -optical storage, tape, 
optical storage, dynamic random access memory, static RAM, flash memory, etc. 

The machine readable storage medium 400 is shown storing a GUI routine 
402, which, when executed, provides a GUI for display of capture and process 
information regarding the time based stream of information, such as the GUI 230 
shown in Figure 3. The GUI routine 402 includes a number of mechanisms for 
playing or otherwise processing the time based stream of information. For example, a 
show capture information routine 404 provides capture information, including a 
capture output from the time based stream of information onto a portion of the 
display, e.g. monitor window 146, in response to information being acquired and 
according to control element commands, as described with reference to Figure 3. A 
show process information routine 406 provides process information and show control 
elements 408 permits control elements to be visible on the display. 

The medium also has a source detector routine 410 for conducting checks for 
communication with the information source. The detector routine 410 may further be 
configured for directing the GUI routine automatically switch the system to a capture 
mode and to provide capture information in response to detecting a communication 
with an information source. The detector routine is associated with the detector 24 as 
described in reference to Figure 2. 

The machine readable storage medium 400 may also store an interrupt 
procedure 414, as described with reference to Figure 4B, for suspending and 
activating a capture procedure. An application procedure 416 may have various 
subroutines for processing control elements, such as for manipulating information and 
constructing a presentation and for executing side operations. 

Other software components may include an operating system 320 and a 
presentation transfer routine 422 that facilitates communication of information from 
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the processing system, such as via external player 84. Various modes of transfer are 
described above with reference to the external player 84 in Figure 1. 

The software components may be provided in as a series of computer readable 
instructions that may be embodied as data signals in a carrier wave. When the 
5 instructions are executed, they cause a processor, such as through a browser to 
perform the capture and editing steps as described. For example, the instructions may 
cause a processor to communicate with an information source, provide capture 
information from the time based stream of information on a first portion of a display, 
provide process information for constructing the presentation on a second portion of 

10 the display; and provide at least one enabled control element. Such instructions may 
be presented to the processor by various mechanisms, such as a plug-in, ActiveX 
control, through use of an application service provided or a network, etc. 

The present invention has been described above in varied detail by reference 
to particular embodiments and figures. However, these specifics should not be 

15 construed as limitations on the scope of the invention, but merely as illustrations of 
some of the presently preferred embodiments. It is to be further understood that other 
modifications or substitutions may be made to the described information transfer 
system as well as methods of its use without departing from the broad scope of the 
invention. Therefore, the following claims and their legal equivalents should 

20 determine the scope of the invention. 
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CLAIMS 



What is claimed is: 



1 1. A method for collecting a time based stream of information in a processing 

2 system for generating a presentation, the method comprising: 

3 A) communicating with an information source having a time based 

4 stream of information; 

5 B) presenting capture information from the time based stream of 

6 information on a portion of a display; 

7 C) presenting process information for constructing the presentation 

8 on the display; and 

9 D) presenting at least one enabled control element. 

10 2. The method of claim 1, further including capturing the time based stream of 

1 1 information from the information source. 

12 3 . The method of claim 2, wherein the capturing is by an interrupt procedure. 

13 4. The method of claim 3, wherein the interrupt procedure iterates at the same 

14 rate or substantially the same rate as the transfer rate of the time based stream 

15 of information. 

16 5. The method of claim 1, wherein at least one of the enabled control element is 

17 to edit the information. 

18 6. The method of claim 1 , wherein at least one of the enabled control elements is 

19 to perform side operations. 

20 7. The method of claim 1 , wherein at least one of the enabled control elements is 

21 an output control. 
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1 8. The method of claim 1, wherein the capture information includes a capture 

2 output presented at the same rate or substantially the same rate as the transfer 

3 rate for the time based stream of information. 

4 9. The method of claim 1 , further including presenting an edit output on the same 

5 portion of the display for presenting of capture information. 

6 10. The method of claim 1, wherein the presenting of capture information is 

7 automatic in response to the communicating with the information source. 

8 11. A processing system for generating a presentation of a time based stream of 

9 information, the system comprising: 

10 A) a capture port for acquiring the time based stream of information; 

11 B) a display device; and 

12 C) a processor to: 

13 i) communicate with an information source having a time 

14 based stream of information through the capture port; 

15 ii) present capture information from the time based stream of 

1 6 information on a portion of the display device; 

1 7 iii) present process information for constructing thepresentation 

18 on the display device; and 

19 iv) present at least one enabled control element. 

20 12. The system of claim 11, wherein the processor is further to capture the time 

21 based stream of information from the information source. 

22 13. The system of claim 12, wherein the capturing is by the processor executing 

23 an interrupt procedure. 

24 14. The system of claim 13, wherein the interrupt procedure iterates at the same 

25 rate or substantially the same rate as the transfer rate of the time based stream 

26 of information. 
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1 15. The system of claim 1 1 , wherein at least one of the enabled control elements is 

2 to edit the information. 

3 1 6. The system of claim 1 1 , wherein at least one of the enabled control elements is 

4 to perform side operations. 

5 17. The system of claim 11, wherein the capture information includes a capture 

6 output presented the same rate or at substantially the same rate as the transfer 

7 rate for the time based stream of information. 

8 18. The system of claim 11, wherein the processor is further to present an edit 

9 output on the same portion of the display for presenting the capture 

10 information. 

11 19. The system of claim 1 1 , wherein the presenting of capture information is 

12 automatic in response to the communicating with the information source. 

13 20. The processing system for collecting a time based stream of information to 

14 generate a presentation comprising: 

1 5 (i) means for communicating with an information source having a 

1 6 time based stream of information; 

17 (ii) means for presenting capture information from the time based 

1 8 stream of information on a portion of the display device; 

19 (iii) means for presenting process information for constructing the 

20 presentation on the display device; and 

21 (iv) means for presenting at least one enabled control element. 

22 21. The system of claim 20, further including a means for capturing the time based 

23 stream of information from the information source. 

24 22. The system of claim 21, wherein the means for capturing is by executing an 

25 interrupt procedure. 
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1 23. The system of claim 22, wherein the interrupt procedure iterates at the same or 

2 substantially the same rate as the transfer rate of the time based stream of 

3 information from the information source 

4 24. The system of claim 20, wherein at least one of the enabled control elements is 

5 to edit the information. 

6 25. The system of claim 20, wherein at least one of the enabled control elements is 

7 to perform side operations. 

8 26. The system of claim 20, further including a means for presenting an edit 

9 output on the same portion of the display for presenting the capture 

1 0 information. 

1 1 27. The system of claim 20, wherein the presenting of capture information is 

12 automatic in response to the communicating with the information source 

13 28. A computer readable medium having stored therein a plurality of sequences of 

14 executable instructions, which, when executed by a processing system for 

1 5 collecting a time based stream of information and generating a presentation, 

1 6 cause the processor to : 

17 A) communicate with an information source having a time based 

1 8 stream of information; 

19 B) provide capture information from the time based stream of 

20 information on a portion of a display; 

21 C) provide process information for constructing the presentation 

22 on the display; and 

23 D) provide at least one enabled control element. 

24 29. The computer readable medium of claim 28, further including additional 

25 sequences of executable instructions, which, when executed by the processor, 

26 cause the processor to capture the time based stream of information from the 

27 information source. 
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1 30. The computer readable medium of claim 28, wherein the capturing is by an 

2 interrupt procedure. 

3 31. The computer readable medium of claim 30, wherein the interrupt procedure 

4 iterates at the same or substantially the same rate as the transfer rate of the 

5 time based stream of information. 

6 32. The computer readable medium of claim 28, wherein the wherein at least one 

7 of the enabled control element is to edit the information. 

8 33. The computer readable medium of claim 28, wherein the at least one of the 

9 enabled control elements is to perform side operations. 

10 34. The computer readable medium of claim 28, wherein the capture information 

1 1 includes a capture output provided at the same rate or substantially the same 

12 rate as the transfer rate for the time based stream of information. 

13 35. The computer readable medium of claim 28, further including additional 

14 sequences of executable instructions, which, when executed by the processor, 

15 cause the processor to provide an edit output on the same portion of the 

1 6 display for presenting the capture information. 

17 36. The computer readable medium of claim 28, wherein the presenting of capture 

18 information is automatic in response to the communicating with the 

19 information source. 

20 37. A method for collecting a time based stream of information in a processing 

21 system for generating a presentation, the method comprising: 

22 A) detecting an information source having a time based stream of 

23 information in communication with the processing system, and 

24 B) automatically presenting capture information from the time 

25 based stream of information on a display in response to the 

26 detecting. 
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1 38. The method of claim 37, further including automatically checking for the 

2 information source in communication with the processing system. 

3 39. The method of claim 37, wherein the detecting is by receiving a signal from 

4 the information source through a capture port on the processing system. 

5 40. The method of claim 37, further including capturing the time based stream of 

6 information from the information source. 

7 41. The method of claim 37, wherein the capture information includes a capture 

8 output provided at the same rate or substantially the same rate as the transfer 

9 rate for the time based stream of information. 

10 42. A processing system for generating a presentation of a time based stream of 

1 1 information, the system comprising: 

12 A) a capture port for acquiring the time based stream of 

13 information; 

14 B) a display device; and 

15 C) a processor to: 

16 i) detect an information source having a time 

17 based stream of information in communication 

1 8 with the processing system, and 

19 ii) automatically present capture information from 

20 the time based stream of information on a 

2 1 display in response to detecting. 

22 43. The system of claim 41, wherein the processor is further to automatically 

23 check for the information source in communication with the processing 

24 system. 

25 44. The system of claim 41, wherein the detecting is by receiving a signal from 

26 the information source through a capture port on the processing system. 
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1 45. The system of claim 41, wherein the processor is further to capture the time 

2 based stream of information from the information source. 

3 46. The system of claim 41, wherein the capture information includes a capture 

4 output provided at the same rate or substantially the same rate as the transfer 

5 rate for the time based stream of information. 

6 47. The processing system for collecting a time based stream of information to 

7 generate a presentation comprising: 

8 A) means for detecting an information source having a time based 

9 stream of information in communication with the processing 

10 system, and 

11 B) means for automatically presenting capture information from 

12 the time based stream of information on a display in response 

13 to detecting. 

14 48. The system of claim 47, further including a means for automatically checking 

15 for the information source in communication with the processing system. 

16 49. The system of claim 47, wherein the detecting is by receiving a signal from 

17 the information source through a capture port on the processing system. 

18 50. The system of claim 47, further including a means for capturing the time based 

1 9 stream of information from the information source. 

20 51. The system of claim 47, wherein the capture information includes a capture 

21 output provided at the same rate or substantially the same rate as the transfer 

22 rate for the time based stream of information. 

23 52. A computer readable medium having stored therein a plurality of sequences of 

24 executable instructions, which, when executed by a processing system for 

25 collecting a time based stream of information and generating a presentation, 

26 cause the processor to: 

27 A) detect an information source having a time based stream of 

28 information in communication with the processing system, and 
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1 

2 



B) automatically present capture information from the time based 
stream of information on a display in response to detecting. 



3 53. The computer readable medium of claim 52, further including additional 

4 sequences of executable instructions, which, when executed by the processor, 

5 cause the processor to automatically check for the information source in 

6 communication with the processing system. 

7 54. The computer readable medium of claim 52, wherein the detecting is by 

8 receiving a signal from the information source through a capture port on the 

9 processing system. 

10 55. The computer readable medium of claim 52, further including additional 

1 1 sequences of executable instructions, which, when executed by the processor, 

12 cause the processor to capture the time based stream of information from the 

13 information source. 

14 56. The computer readable medium of claim 52, wherein the capture information 

1 5 includes a capture output provided at the same rate or substantially the same 

16 rate as the transfer rate for the time based stream of information. 

17 57. A method for generating a presentation of a time based stream of information 

18 in a processing system, the method comprising: 

19 A) capturing the time based stream of information from an 

20 information source into the processing system during a capture 

21 mode; 

22 B) presenting a capture output on a viewing portion of a display 

23 during the capture mode; and 

24 C) presenting an edit output on the viewing portion of the display 

25 during an edit mode. 
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1 

2 



58. 



The method of claim 57, wherein the presenting of the capture output is at the 
same rate or substantially the same rate as the transfer rate for the time based 
stream of information. 



4 59. The method of claim 57, further including providing at least one enabled 

5 control element during the capture mode and edit mode. 

6 60. The method of claim 59, wherein at least one of the enabled control element 

7 includes a control element perform side operations. 

8 61. A processing system for generating a presentation of a time based stream of 

9 information, the system comprising: 

10 A) a capture port for acquiring the time based stream of 

1 1 information; 

12 B) a display device; and 

13 C) a processor to: 

14 i) capture the time based stream of information 

15 from an information source into the processing 

1 6 system during a capture mode; 

17 ii) present a capture output on a viewing portion of 

18 a display during the capture mode; and 

1 9 iii) present an edit output on the viewing portion of 

20 the display during an edit mode. 

21 62. The system of 61, wherein the presenting of the capture output is at the same 

22 rate or substantially the same rate as the transfer rate for the time based stream 

23 of information. 

24 63. The system of claim 61, wherein the processor is further to provide at least 

25 one enabled control element during the capture mode and edit mode. 

26 64. The system of claim 63, wherein at least one of the enabled control element is 

27 to perform side operations. 
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1 65. A processing system for collecting a time based stream of information to 

2 generate a presentation comprising: 

3 A) means for capturing the time based stream of information from 

4 an information source into the processing system during a 

5 capture mode; 

6 B) means for presenting a capture output on a viewing portion of a 

7 display during the capture mode; and 

8 C) means for presenting an edit output on the viewing portion of 

9 the display during an edit mode. 

1 0 66. The system of claim 65, wherein the means for presenting the capture output is 

1 1 for presenting at the same rate or substantially the same rate as the transfer rate 

1 2 for the time based stream of information. 

13 67. The system of claim 65, further including a means for providing at least one 

14 enabled control element during the capture mode and edit mode. 

1 5 68. The system of claim 67, wherein at least one of the enabled control element is 

16 to perform side operations. 

17 69. A computer readable medium having stored therein a plurality of sequences of 

18 executable instructions, which, when executed by a processing system for 

19 collecting a time based stream of information and generating a presentation, 

20 cause the processor to: 

21 A) capture the time based stream of information from an 

22 information source into the processing system during a capture 

23 mode; 

24 B) present a capture output on a viewing portion of a display 

25 during the capture mode; and 

26 C) present an edit output on the viewing portion of the display 

27 during an edit mode. 
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1 70. The computer readable medium of claim 69, wherein the presenting of the 

2 capture output is at the same rate or substantially the same rate as the transfer 

3 rate for the time based stream of information. 

4 71. The computer readable medium of claim 69, further including additional 

5 sequences of executable instructions, which, when executed by the processor, 

6 cause the processor to provide at least one enabled control element during the 

7 capture mode and edit mode. 

8 72. The computer readable medium of claim 71, wherein at least one of the 

9 enabled control element is to perform side operations. 

10 73. A method of collecting a time based stream of information from an editing 

1 1 window in a processing system, the method comprising: 

12 A) detecting the coupling of an information source to the processing 

13 system, and 

14 B) automatically engaging a capture mode. 

15 74. The method of claim 73, further including presenting a captured time based 

1 6 stream of information in the editing window. 

17 75. The method of claim 73, wherein the editing window includes a toggle control 

1 8 element to switch between capture and edit mode within the editing window. 

19 76. A processing system for collecting a time based stream of information from an 

20 editing window, the system comprising: 

21 A) a capture port for acquiring the time based stream of information; 

22 B) a display device; and 

23 C) a processor to: 

24 i) detect the coupling of an information source to the processing 

25 system, and 

26 ii) automatically engage a capture mode. 
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1 77. The system of claim 76, wherein the processor is further to present a captured 

2 time based stream of information in the editing window. 

3 78. The system of claim 76, wherein the editing window includes a toggle control 

4 element to switch between capture and edit mode within the editing window. 

5 79. A processing system for collecting a time based stream of information from an 

6 editing window comprising: 

7 A) a means for detecting the coupling of an information source to the 

8 processing system, and 

9 B) a means for automatically engaging a capture mode. 

10 80. The system of claim 79, further including a means for presenting a captured 

1 1 time based stream of information in the editing window. 

12 81. The system of claim 79, wherein the editing window includes a toggle control 

13 element to switch between capture and edit mode within the editing window. 

14 82. A computer readable medium having stored therein a plurality of sequences of 

15 executable instructions, which, when executed by a processing system for 

16 collecting a time based stream of information and generating a presentation, 

1 7 cause the processor to: 

18d A) detect the coupling of an information source to the processing 

19 system, and 

20 B) automatically engage a capture mode. 

21 83. The computer readable medium of claim 82, further including additional 

22 sequences of executable instructions, which, when executed by the processor, 

23 cause the processor to present a captured time based stream of information in 

24 the editing window. 
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1 84. The computer readable medium of claim 82, wherein the editing window 

2 includes a toggle control element to switch between capture and edit mode 

3 within the editing window. 
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ABSTRACT 

The method of collecting a time based stream of information to generate a 
presentation is provided for in which a processing system is employed. In one 
exemplary method, the processing system communicates with an information source 

5 that contains a time based stream of information. This communication may be through 
an automatic detection procedure that may trigger a capture mode and the gathering of 
the time based stream of information. A user interface may present controls for 
toggling between capture mode and edit mode within a single window of the user 
interface. A capture output, which is often a substantially true representation of the 

1 0 quality of information being acquired, may be provided on a portion of the display, 
e.g. viewing portion, during the capture mode. An edit output that represents the 
edited version of clips may be provided on the same viewing portion of the display 
during the edit mode. The use of an interrupt procedure allows for controls to be 
enabled during capture such that the user may perform various tasks while the system 

15 is loading the time based stream of information. Other aspect of the present invention 
relating to the processing system for collecting a time based stream of information for 
use in authoring a presentation are also described. 
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FIGURE 4B 
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Attorney's Docket No.: 004860. P2471 Patent 
DECLARATION AND POWER OF ATTORNEY FOR PATENT APPLICATION 



As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below, next to my name. 

I believe I am the original, first, and sole inventor (if only one name is listed below) or an original, first, and 
joint inventor (if plural names are listed below) of the subject matter which is claimed and for which a 
patent is sought on the invention entitled 

AUTOMATIC CAMERA CAPTURE MODE/SINGLE WINDOW 
FOR CAPTURE AND EDITING 



the specification of which 

X is attached hereto. 

was filed on as 

United States Application Number 

or PCT International Application Number 

and was amended on . 

(if applicable) 

I hereby state that I have reviewed and understand the contents of the above-identified specification, 
including the claim(s), as amended by any amendment referred to above. I do not know and do not 
believe that the claimed invention was ever known or used in the United States of America before my 
invention thereof, or patented or described in any printed publication in any country before my invention 
thereof or more than one year prior to this application, that the same was not in public use or on sale in 
the United States of America more than one year prior to this application, and that the invention has not 
been patented or made the subject of an inventor's certificate issued before the date of this application in 
any country foreign to the United States of America on an application filed by me or my legal 
representatives or assigns more than twelve months (for a utility patent application) or six months (for a 
design patent application) prior to this application. 

! acknowledge the duty to disclose all information known to me to be material to patentability as 
defined in Title 37, Code of Federal Regulations, Section 1.56. 

[ hereby claim foreign priority benefits under Title 35, United States Code, Section 1 19(a)-(d), of any 
foreign application (s) for patent or inventor's certificate listed below and have also identified below 
any foreign application for patent or inventor's certificate having a filing date before that of the 
application on which priority is claimed: 
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Prior Foreign Application(s) 



Priority 
Claimed 



(Number) (Country) (Day/Month/Year Filed) Yes No 



(Number) (Country) (Day/Month/Year Filed) Yes No 



(Number) (Country) (Day/Month/Year Filed) Yes No 

I hereby claim the benefit under title 35, United States Code, Section 1 19(e) of any United States 
provisional application(s) listed below: 



(Application Number) Filing Date 



(Application Number) Filing Date 



I hereby claim the benefit under Title 35, United States Code, Section 120 of any United States 
applications) listed below and, insofar as the subject matter of each of the claims of this application 
is not disclosed in the prior United States application in the manner provided by the first paragraph 
of Title 35, United States Code, Section 112, I acknowledge the duty to disclose all information 
known to me to be material to patentability as defined in Title 37, Code of Federal Regulations 
Section 1 .56 which became available between the filing date of the prior application and the national 
or PCT international filing date of this application: 



(Application Number) Filing Date (Status - patented, 

pending, abandoned) 



(Application Number) Filing Date (Status - patented, 

pending, abandoned) 

I hereby appoint the persons listed on Appendix A hereto (which is incorporated by reference and a 
part of this document) as my respective patent attorneys and patent agents, with full power of 
substitution and revocation, to prosecute this application and to transact all business in the Patent 
and Trademark Office connected herewith. 

Send correspondence to Lisa N. Benado . BLAKELY, SOKOLOFF, TAYLOR & 

(Name of Attorney or Agent) 
ZAFMAN LLP, 12400 Wilshire Boulevard 7th Floor, Los Angeles, California 90025 and direct 
telephone calls to Lisa N. Benado . (408) 720-8598. 

(Name of Attorney or Agent) 
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I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these 
statements were made with the knowledge that willful false statements and the like so made 
are punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United 
States Code and that such willful false statements may jeopardize the validity of the 
application or any patent issued thereon. 



Full Name of Sole/First Inventor Glenn Reid 



Inventor's Signature Date 

Residence Citizenship 

(City, State) (Country) 

Post Office Address 



Full Name of Second/Joint inventor 



Inventor's Signature Date 

Residence Citizenship 



(City, State) (Country) 
Post Office Address 



Full Name of Third/Joint Inventor 



Inventor's Signature Date 

Residence Citizenship 



(City, State) (Country) 

Post Office Address 



Full Name of Fourth/Joint Inventor 



Inventor's Signature Date 

Residence Citizenship , 



(City, State) (Country) 
Post Office Address 
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APPENDIX A 



William E. Alford, Reg. No. 37,764; Farzad E. Amini, Reg. No. P42.261; Aloysius T. C. AuYeung, Reg. No. 
35,432; William Thomas Babbitt, Reg. No. 39,591; Carol F. Barry, Reg. No. 41,600; Jordan Michael 
Becker, Reg. No. 39,602; Bradley J. Bereznak, Reg. No. 33,474; Michael A. Bernadicou, Reg. No. 35,934; 
Roger W. Blakely, Jr., Reg. No. 25,831 ; Gregory D. Caldwell, Reg. No. 39,926; Andrew C. Chen, Reg. No. 
43,544; Thomas M. Coester, Reg. No. 39,637; Dennis M. deGuzman, Reg. No. 41,702; Stephen M. De 
Klerk, under 37 C.F.R. § 10.9(b); Michael Anthony DeSanctis, Reg. No. 39,957; Daniel M. De Vos, Reg. 
No. 37,813; Robert Andrew Diehl, Reg. No. 40,992; Sanjeet Dutta, Reg. No. P46,145; Matthew C. Fagan, 
Reg. No. 37,542; Tarek N. Fahmi, Reg. No. 41,402; Paramita Ghosh, Reg. No. 42,806; James Y. Go, 
Reg. No. 40,621; James A. Henry, Reg. No. 41,064; Willmore F. Holbrow III, Reg. No. P41.845; Sheryl 
Sue Holloway, Reg. No. 37,850; George W Hoover II, Reg. No. 32,992; Eric S. Hyman, Reg. No. 30,139; 
William W. Kidd, Reg. No. 31 ,772; Sang Hui Kim, Reg. No. 40,450; Eric T. King, Reg. No. 44,188; Kurt P. 
Leyendecker, Reg. No. 42,799; Michael J. Mallie, Reg. No. 36,591; Andre L. Marais, under 37 C.F.R. § 
10.9(b); Paul A. Mendonsa, Reg. No. 42,879; Chun M. Ng, Reg. No. 36,878; Thien T. Nguyen, Reg. No. 
43,835; Thinh V. Nguyen, Reg. No. 42,034; Dennis A. Nicholls, Reg. No. 42,036; Daniel E. Ovanezian, 
Reg. No. 41,236; Marina Portnova, Reg. No. P45.750; Babak Redjaian, Reg. No. 42,096; William F. 
Ryann, Reg. 44,313; James H. Salter, Reg. No. 35,668; William W. Schaal, Reg. No. 39,018; 
James C. Scheller, Reg. No. 31 ,195; Jeffrey Sam Smith, Reg. No. 39,377; Maria McCormack Sobrino, 
Reg. No. 31 ,639; Stanley W. Sokoloff, Reg. No. 25,128; Judith A. Szepesi, Reg. No. 39,393; Vincent P. 
Tassinari, Reg. No. 42,179; Edwin H. Taylor, Reg. No. 25,129; John F. Travis, Reg. No. 43,203; Joseph 
A. Twarowski, Reg. No. 42,191; Lester J. Vincent, Reg. No. 31,460; Glenn E. Von Tersch, Reg. No. 
41 ,364; John Patrick Ward, Reg. No. 40,216; Mark L. Watson, Reg. No. P46,322; Thomas C. Webster, 
Reg. No. P46,154; Charles T. J. Weigell, Reg. No. 43,398; Kirk D. Williams, Reg. No. 42,229; Steven D. 
Yates, Reg. No. 42,242; and Norman Zafman, Reg. No. 26,250; my patent attorneys, and Justin M. Dillon, 
Reg. No. 42,486; my patent agent, of BLAKELY, SOKOLOFF, TAYLOR & ZAFMAN LLP, with offices 
located at 12400 Wilshire Boulevard, 7th Floor, Los Angeles, California 90025, telephone (310) 207-3800, 
and James R. Thein, Reg. No. 31,710, my patent attorney. I also hereby appoint Albert P. Cefalo, Reg. 
No. 27,315; Mark Aaker, Reg. No. 32,667, Richard Liu, Reg. No. 34,377; Helene Plotka Workman, Reg. 
No. 35,981 ; and Edward W. Scott, IV, Reg. No. 36,000; my attorneys; of APPLE COMPUTER, INC., 
located at 1 Infinite Loop, MS: 3-PAT, Cupertino, California 95014, telephone (408)974-9453, will full 
power of substitution and revocation, to prosecute this application and to transact all business in the 
Patent and Trademark Office connected herewith. 
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APPENDIX B 



Title 37, Code of Federal Regulations, Section 1.56 
Duty to Disclose Information Material to Patentability 

(a) A patent by its very nature is affected with a public interest. The public interest is best served, 
and the most effective patent examination occurs when, at the time an application is being examined, the 
Office is aware of and evaluates the teachings of all information material to patentability. Each individual 
associated with the filing and prosecution of a patent application has a duty of candor and good faith in 
dealing with the Office, which includes a duty to disclose to the Office all information known to that individual 
to be material to patentability as defined in this section. The duty to disclosure information exists with respect 
to each pending claim until the claim is cancelled or withdrawn from consideration, or the application becomes 
abandoned. Information material to the patentability of a claim that is cancelled or withdrawn from 
consideration need not be submitted if the information is not material to the patentability of any claim 
remaining under consideration in the application. There is no duty to submit information which is not material 
to the patentability of any existing claim. The duty to disclosure all information known to be material to 
patentability is deemed to be satisfied if all information known to be material to patentability of any claim 
issued in a patent was cited by the Office or submitted to the Office in the manner prescribed by §§1.97(b)-(d) 
and 1.98. However, no patent will be granted on an application in connection with which fraud on the Office 
was practiced or attempted or the duty of disclosure was violated through bad faith or intentional misconduct. 
The Office encourages applicants to carefully examine: 

(1) Prior art cited in search reports of a foreign patent office in a counterpart application, and 

(2) The closest information over which individuals associated with the filing or prosecution of a 
patent application believe any pending claim patentably defines, to make sure that any material information 
contained therein is disclosed to the Office. 

(b) Under this section, information is material to patentability when it is not cumulative to 
information already of record or being made or record in the application, and 

(1) It establishes, by itself or in combination with other information, a prima facie case of 
unpatentability of a claim; or 

(2) It refutes, or is inconsistent with, a position the applicant takes in: 

(i) Opposing an argument of unpatentability relied on by the Office, or 

(ii) Asserting an argument of patentability. 

A prima facie case of unpatentability is established when the information compels a conclusion that a claim is 
unpatentable under the preponderance of evidence, burden-of-proof standard, giving each term in the claim 
its broadest reasonable construction consistent with the specification, and before any consideration is given to 
evidence which may be submitted in an attempt to establish a contrary conclusion of patentability. 

(c) Individuals associated with the filing or prosecution of a patent application within the 
meaning of this section are: 

(1 ) Each inventor named in the application; 

(2) Each attorney or agent who prepares or prosecutes the application; and 

(3) Every other person who is substantively involved in the preparation or prosecution of the 
application and who is associated with the inventor, with the assignee or with anyone to whom there is an 
obligation to assign the application. 

(d) Individuals other than the attorney, agent or inventor may comply with this section by 
disclosing information to the attorney, agent, or inventor. 
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